Individual differences in response times

.title[
# Individual differences in response times
]
.author[
### Adriana F. Chávez De la Peña
]
.date[
### April 10, 2023
]

---

# Outline

1.- The study of individual differences.

2.- Individual differences in cognitive control.

3.- Response time distribution analysis

4.- Cognitive latent variable modeling of response times

5.- Research proposals

---

# Outline

**1.- The study of individual differences.**

We discuss the differences between experimental and differential psychology, with an emphasis on how they treat between-subject variability.

2.- Individual differences in cognitive control.

3.- Response time distribution analysis

4.- Cognitive latent variable modeling of response times

5.- Research proposals

---

# Outline

1.- The study of individual differences.

**2.- Individual differences in cognitive control.**

We discuss the specific case of cognitive control, where correlations across dependent measures obtained through experimental paradigms have been found to be weak.

3.- Response time distribution analysis

4.- Cognitive latent variable modeling of response times

5.- Research proposals

---

# Outline

1.- The study of individual differences.

2.- Individual differences in cognitive control.

**3.- Response time distribution analysis**

We suggest a shift from correlational studies onto response time distribution analysis. We review models commonly used when working with response time (RT) data, distinguishing between simple RT models and choice-RT models.

4.- Cognitive latent variable modeling of response times

5.- Research proposals

---

# Outline

1.- The study of individual differences.

2.- Individual differences in cognitive control.

3.- Response time distribution analysis

**4.- Cognitive latent variable modeling of response times**

We talk about cognitive psychometrics and cognitive latent variable models (CLVM) as a sophisticated tool with the potential to unveil the underlying structure of parameters retrieved from RT models.

5.- Research proposals

---

# Outline

1.- The study of individual differences.

2.- Individual differences in cognitive control.

3.- Response time distribution analysis

4.- Cognitive latent variable modeling of response times

**5.- Research proposals**

---

# 1. The study of individual differences

---

## Two disciplines in psychology (Cronbach, 1957)

**Experimental research:** Meticulous variable manipulations are implemented in search of specific effects.

- **Q:** How does performance changes between *groups*? 
  
--
  
- ANOVA-based

- Experimental design aims to minimize between-subjects variance in order to attribute the results observed to the manipulations imposed by the experimenter only.

**Differential research:** Captures the individual variability observed in the data.

- **Q:** How do people differ from one another?

- Correlation-based.

- Individual-level traits are captured by the correlations between measurements taken across subjects.

---

## Two disciplines in psychology (Cronbach, 1957)

**Experimental research:** Meticulous variable manipulations are implemented in search of specific effects.

- **Q:** How does performance changes between *groups*?

- ANOVA-based

- **Experimental design aims to minimize between-subjects variance in order to attribute the results observed to the manipulations imposed by the experimenter only.**

**Differential research:** Captures the individual variability observed in the data.

- **Q:** How do people differ from one another?

- Correlation-based.

- **Individual-level traits are captured by the correlations between measurements taken across subjects.**

---

## Two disciplines in psychology (Cronbach, 1957)

**Experimental research:** Meticulous variable manipulations are implemented in search of specific effects.

- **Q:** How does performance changes between *groups*?

- ANOVA-based

- **Reduce between-subject variance**

**Differential research:** Captures the individual variability observed in the data.

- **Q:** How do people differ from one another?

- Correlation-based.

- **Maximize between-subjects variance**

---

## Reliability

- We talk about reliability to refer to the degree to which we can reliably obtain the same results under similar conditions.

- How *reliable* are our findings?

- Typically quantified in terms of the correlation between related measures (i.e., parallel measures; Lord et al., 1968).

---

## Reliability

- An experimental effect is reliable to the extent that similar findings are reported consistently across replicate studies.

- A measurement is reliable as long as it consistently rank-orders individuals across applications.

The closer the performance observed across participants...

… the better, for an experimental researcher.

… the worse, for a differential researcher.

**Robust experimental effects do not necessarily imply reliable individual differences.**

---

### An example by Logie et al. (1996)

A study to explore the reliability of the **word-length effect** and the **phonological-similarity effect** on a serial order recall task.

**First experiment**  ( `$N=251$` )

- Strong aggregate effects (group mean differences),

**Retest ** ( `$N=40$` )

- Again, strong aggregate effects.

- Low correlations (r = 0.1 - 0.3) across experiments.

**Whether a subject showed the effect in the initial test was not predictive of their performance on retest.**

---

## Reliability paradox (Hedge et al., 2018)

- Experimental tasks minimize between-subject variability, which restrains the correlations one can compute from these measures.

- Robust experimental effects do not necessarily imply reliable individual differences.

- Low correlations found between related task-derived measures present a challenge for differential researchers.

---

# 2. Individual differences in cognitive control

---

# Cognitive control

The ability to ignore irrelevant information and to suppress prepotent responses when solving a task (Logan & Cowan, 1984).

**EXAMPLE: STROOP TASK**

- YELLOW

- GREEN

- RED

- BLUE

**Two experimental conditions:** Congruent / Incongruent

---

# Cognitive control

There are three main executive functions (Miyake et al., 2000):

1) Shifting between mental sets or tasks.

2) Controlling inputs and responses considered when solving a task.

3) Monitoring and updating the contents of working memory.

---

# Cognitive control

There are three main executive functions (Miyake et al., 2000):

1) Shifting.

**2) Cognitive control.**

3) Updating.

Cognitive control has two main components (Friedman & Miyake, 2004):

1) Response-distracter inhibition

2) Resistance to proactive interference

---

## Cognitive control tasks

### Flanker task

---

## Cognitive control tasks

### Global/local task

<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#./Pictures/globalLocal_incongruent.png" alt="Andres and Fernandes, 2006" width="50%" />
Andres and Fernandes, 2006
</div>

---

## Cognitive control tasks

### Simon task

<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#./Pictures/simonnnnn.png" alt="Kharitonova et al. (2013)" width="95%" />
Kharitonova et al. (2013)
</div>

---

## Cognitive control tasks

### Antisaccade

<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#./Pictures/antisaccade.png" alt="Everling and Fischer (1998)" width="50%" />
Everling and Fischer (1998)
</div>

---

## Cognitive control tasks

Common elements...

- Distinction between “congruent” and "incongruent" trials.

- The core assumption is that participants should take longer to respond to incongruent trials.

- Cognitive control is measured in terms of differences in response times (RT) observed across conditions following **Donders' subtraction method**

- Data cleaning often involves removing extreme RTs and trials where an error was made.

---

## Donders' subtraction method (1868)

- Mental processes do not occur simultaneously.

- Distinctive mental processes can be isolated by subtracting the **mean response time** observed across different conditions.

---

## Individual differences in cognitive control

- What can we learn about one person's cognitive control from any given cognitive control task?

- Can we predict performance on task B from the score observed on task A?

- What are the **individual differences in cognitive control**?

---

## Individual differences in cognitive control

(Rey-Mermet et al., 2019)

---

## Individual differences in cognitive control

---

## Individual differences in cognitive control

---

## Big debate: What do these low correlation imply?

**1.- A substantive problem:** Is cognitive control a cognitive process that can be studied as an individual trait or not?

**2.- A measurement challenge:** Does it make sense to use difference scores to study individual differences in cognitive control?

---

## The measurement challenge

Correlational studies are conducted to capture individual differences in a given trait of interest under two core assumptions:

- 1) We are using scores related to the same process or to different components of a unitarian process.

- 2) We are using scores that provide reliable information.

**Task-specific difference scores need to be highly reliable.**

---

## Reliability of difference scores

`$$\Huge \rho_{dd'} = \frac{\rho_{xx'}-\rho_{xy}}{1-\rho_{xy}}$$`

Lord (1963), cited by Draheim et al. (2019)

Let's assume...

...Individual components have perfect reliability ( `$\large \rho_{xx'}=1$` )

...Individual components are independent from each other ( `$\large \rho_{xy}=0$` )

---

## Reliability of difference scores

- Hedge et al. (2018) conducted three test-retest studies. The most reliable cognitive control task was found to be the Stroop task ( `$0.60 - 0.66$` )

- Enkavi et al. (2019) summarize reliability indices reported in the literature, which were found to be highly variable (i.e., the Stroop task  varied between 0.27 and 0.8, and the Simon task, oscillated between the whole `$0-1$` interval).

---

## Reliability of difference scores

- High correlations between incongruent and congruent RT are expected.

- It is expected to have shared systematic variance across incongruent and congruent RTs.

- Subtracting one of them from the other removes the systematic variance, leaving only the error variance.

---

## Factorial structure of overall RT

**Rey-Mermet et al., 2019**

---

## Factorial structure of difference scores

**Rey-Mermet et al., 2019**

---

## Arguments against difference scores

- “Differences between scores are much more unreliable than the scores themselves” (Lord, 1956).

- “Scores formed by subtracting pretest scores from postest scores lead to fallacious conclusions, primarily because such scores are systematically related to any random error of measurement” (Cronbach and Furby, 1970)

- “Difference scores have low convergent validity caused by deficiencies in test-retest reliability” (Paap & Sawi, 2016)

**Overall:** Difference scores are particularly problematic for differential research.

---

## What are the alternatives?

--

- 1) Switching to accuracy metrics.

- 2) To develop a new metric that incorporates accuracy and RTs.

- 3) A different modeling approach...

---

# 3. Response time modeling

---

##  Response time (RT) modeling

- RTs are known to be highly variable.

- The analysis of RT distributions constitutes a long-standing tradition in Psychology (Hohle, 1965; Luce, 1986; Ratcliff, 1978)

- Specific characteristics of RT distributions are thought to shed light on the psychological processes that underlie the behavior observed.

---

## The Worst performance rule

- Slowest RTs are more indicative of general intelligence than the faster RTs (Larson & Alderton, 1990).

- When individual RTs are ranked and grouped into RT bands, the median slowest RTs correlate better with measures of general intelligence, working memory and processing speed than the fastest median RTs (Coyle, 2003).

- Point estimates such as the mean or median RT neglects the RT variability observed.

---

## Two types of RT models

**1.- Simple RT models**

Mathematical models that describe empirical RT distributions and whose quantitative properties (i.e., shift, scale and shape parameters) can be mapped to specific experimental manipulations (Andrews & Heathcote, 2001; Heathcote et al., 1991; Hockley, 1984; Hohle, 1965; Spieler et al., 2000).

**2.- Choice RT models**

- RTs are conceived as the stopping point of an internal processes generating the choices observed, (Noorani & Carpenter, 2016), and as such, RTs are modeled jointly with the choices observed.

---

## Simple RT models: the exGaussian distribution

---

## Simple RT models: the exGaussian distribution

- The exGaussian distribution has proven suitable to model empirical RT distributions (Heathcote et al., 1991; Hockley, 1984; Mewhort et al., 1992; Ratcliff, 1979).

- Parameters related to the normal component have been associated with peripheral sensory–motor and automatic processes

- The exponential component has been found to be related to decision-related processes (Hohle, 1965; Gordon & Carson, 1990; Rohrer & Wixted, 1994).

- The exGaussian parameters have also been mapped to drift diffusion model parameters (e.g., faster drift rates are associated with smaller rates, while more conservative response criteria are linked with larger values of `$\mu$` ; Spieler, 2001; Schmiedek et al., 2007).

---

## Simple RT models: the logNormal distribution

---

## Choice RT models

“...most real-life decisions are composed of two separate decisions: first the decision to stop deliberating and act, and then the decision or act itself”.

Forstmann et al. (2016)

- Sequential sampling models account for the speed-accuracy tradeoff observed (Stone, 1960; Ratcliff & Smith, 2004).

- These sampling models assume that evidence for any response alternative is accumulated over time until a response threshold is reached.

- They account jointly for the responses and RTs observed.

---

### LATER model (Carpenter, 1981)

- The linear approach to threshold with ergodic rate (LATER) model considers only **two free parameters**.

- Founded on the empirical finding that the inverse of the RTs follow a normal distribution (with parameters `$\mu$` and `$\sigma$` ).

- Non-decision time is assumed to be fixed and is otherwise not considered by this model.

---

### LATER model components

- Every stimuli `$S$` has an initial value `$S_0$`

- Internal process progresses linearly with constant rate `$r$`, which varies across trials following a normal distribution.

- Response is initiated when a threshold `$S_T=S_0 + \theta$` is met

- RTs observed are given by `$T=\frac{\theta}{r}$`

- Since `$r \sim \mbox{Normal}(\mu, \sigma)$`, then `$\frac{1}{T}$` is also normally distributed, thus replicating the empirically reported pattern.

---

### LATER model

<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#./Pictures/LATER.png" alt="Noorani and Carpenter, 2016" width="90%" />
Noorani and Carpenter, 2016
</div>

---

### Linear ballistic accumulation model (Brown & Heathcote, 2008)

- An extension of the LATER model that incorporates a parameter to capture non-decision time.

- The accumulation rate is now referred to as drift rate, `$d$`.

- Each response alternative is assumed to have its own independent accumulator.

- The response observed corresponds to the first accumulator to reach the threshold value `$b$`.

- The starting point per accumulator is assumed to vary across trials following a uniform distribution with bounds 0 and `$a$` (such that `$a < b$`)

---

### Linear ballistic accumulation model (Brown & Heathcote, 2008)

<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#./Pictures/LBA.png" alt="Brown and Heathcote, 2008" width="98%" />
Brown and Heathcote, 2008
</div>

---

### Drift diffusion model (Ratcliff, 1978)

<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#./Pictures/DDM_Illustration3.png" alt="Johnson et al., 2017" width="90%" />
Johnson et al., 2017
</div>

---

### Circular drift diffusion model (Smith, 2016)

---

# 4. Cognitive latent variable modeling of response times

---

## Latent variable models and psychometrics

The models described in previous chapters are identified as cognitive models because they account for the psychological processes that underlie the data observed. In contrast, psychometric models, or LVMs, constitute a type of model that assumes that the data observed results from the linear combination of a small number of latent variables that vary across different random levels (i.e., across participants, conditions, tasks, etc.) and some measurement error (Baribault, 2019).

LVMs are used to identify the components that can account for the data observed and their different levels of variation. This type of model allows us to determine whether variability in the data can be attributed to differences between participants or to the effect of specific features of the experimental design (i.e., experimental conditions, stimuli characteristics, levels of an experimental manipulation, etc.).

An important difference between cognitive models and LVMs is that the latter do not assume that every latent variable associated with the data has a relevant psychological interpretation. LVMs are often implemented with the goal of identifying the smallest number of substantially relevant latent components that can be used to account for the majority of the variability contained in the data (Bollen, 2002).

---

## Cognitive psychometrics and individual differences

We use the term cognitive psychometrics to refer to the psychometric treatment of cognitive models (Batchelder, 1998; Batchelder, 2010) with the intention of identifying whether the cognitive processes described by cognitive models vary across participants, items, conditions, tasks, and so on. For example, clinical researchers aim to identify how individuals from different populations may vary in terms of the parameter values estimated from cognitive models used to account for their performance in cognitive tasks, and how these changes may be elicited over time or across conditions.

Using LVM techniques onto parameters associated with a cognitive model presents a reconciliation point among experimental and differential research (Cronbach, 1957; Borsboom, 2006). The main advantage of cognitive psychometrics is that it preserves the best of both worlds: First, parameters derived from cognitive models are known to be informative of the cognitive processes underlying behavior and thus are highly interpretable. Second, we move away from the use of aggregated data to recognize the richness of the information provided by the data variability, considering differences in performance across many possible levels of random variation (Riefer et al., 2002).

As an example of the application of the cognitive psychometrics approach, we discuss the study conducted by Schmiedek et al. (2007). In this study, eight different choice response time tasks were applied along with six tasks used to measure working memory and intelligence. From the data collected, an exGaussian model was fit to the RT data and parameters of the DDM were estimated using both RTs and choice data. Then, the authors performed confirmatory factorial analysis techniques on all parameter values obtained, and report that individual differences in the rate parameter of the exGaussian distribution (i.e., ) were found to be a unique predictor for the individual differences in the performance observed across the different working memory and reasoning tasks included in the study. Along these lines, we can also find studies on the latent variable structure of cognitive control from the perspective of cognitive psychometrics (Miyake et al., 2000; Friedman and Miyake, 2004).

---

## Cognitive latent variable models

The joint approach postulated by cognitive psychometrics allows researchers to make conclusions far more informative than either a cognitive model or an LVM could afford separately. LVMs are regarded as informative of the internal structure of the data, but these types of models provide very limited insights into the underlying processes and mechanisms that could be generating the data, while cognitive models are limited in their ability to identify higher-order latent structures.

Despite the advantages presented by cognitive psychometrics in terms of the enhanced insights we can gain from the data collected about the underlying processes of interest, the applications described in the previous section present some major limitations. The studies conducted by Friedman and Miyake in 2004 and by Schmiedek et al. in 2007 consist of a two-step application of the joint modeling technique postulated by the cognitive psychometrics approach. Both studies perform the parameter estimation from the cognitive model of interest first, and then in a second step, they use latent variable modeling techniques to describe the latent variable structure of these parameter values. This two-step approach is limited because the variability contained in the data is only used to inform the parameter estimation, but it is neglected in the identification of their latent variable structure. Thus, the latent variable structure revealed in the second step considers only the variability in the parameter values recovered, while ignoring the uncertainty contained in the raw data.

As an alternative, Vandekerckhove (2014) introduced CLVMs as a class of models that explore the latent variable structure of the parameters of cognitive models, all as part of a hierarchical Bayesian model. As an illustrative example, the author presents a CLVM that uses the DDM as the main cognitive model, such that by imposing a latent variable model to describe the parameters of the DDM we are now able to draw substantive conclusions about the latent variables that could be explaining the differences observed in processing speed, response caution, and son on, that are made evident if we simply compare the parameter values recovered across different possible levels of variation.

CLVMs allow to make inferences about the unobserved cognitive processes generating the data collected in terms of the influence of many unobserved latent variables. A key feature of CLVM is that both the cognitive model parameters and their latent variable structure is being inferred as part of a single hierarchical Bayesian model. This feature comes with the huge advantage that the uncertainty contained in the data is captured and allocated across both model components (Vandekerckhove, 2014; Baribault, 2019).

---

# 5. Research proposals

---

## 1) Exploring the unidimensionality of processing speed

**Goal:** Explore the dimensionality of processing speed in extended data sets, in contrast with the dimensionality of human anthropometrics.

- Perform a PCA over the processing speed observed across different tasks.

- Perform a PCA over a large anthropometric dataset.

---

## 2) exGaussian CLVM

**Goal:** To explore the unidimensional structure of processing speed across cognitive control tasks through the lens of RT distribution analysis.

- Determine whether this univariance can be tracked back to the shift, scale, or shape parameters of the RT distribution.

---

## 3) Categorical drift-diffusion model

*Goal:* To develop extension of the CDDM where the bounded continuous decision space is segmented into bins.

- Develop a JAGS module.

- Present a paremet recovery study.

- Show an application to real-world data.

---

# IN SUMMARY

1.- The study of individual differences depends on high between-subject variability.

2.- Individual differences in cognitive control have been mostly approximated through task-derived dependent measures.

3.- Response Time distribution analysis presents an alternative to the use of task-derived dependent measures.

4.- We can implement cognitive latent variable models that capture the latent structure of the parameters of a given RT model.

---

# Thank you!